Conversation
There was a problem hiding this comment.
I like the idea of Stochastic state. But this raises the question of having a class for states. It is slightly weird (yet true) that a state is stateful.
In principle, we could have an abstract class State, it would have an abstract static method initial() -> State where the return state is a state of same type as this. I think Stateful would then be generic over the State type (so in python I guess it must take a State class type). We can have two States, one that contains data directly, and another would be stochastic state. The structure isn't clear in my mind, but there may be a path.
|
Well, this is the first time where we need to reuse a state for several different classes (and it may be the only time). So I think we could wait to have a real need for state classes before implementing them. Let's not overengineer the structure too early. For now we have a bigger problem to solve IMO: the generator idea doesn't work on cuda. |
2 participants
@PierreQuinton Just made a draft PR to show you an example of what I had in mind. I only made the implementation for pcgrad.
I also forgot the reset method of PCGrad and PCGradWeighting.